<!DOCTYPE html>
<html lang='en'><head>
  <title>Flask主线程以及短链接实现 | 江小凡的博客</title>
  <meta charset='utf-8'>
  <meta name="generator" content="Hugo 0.76.5" />
  <meta name = 'viewport' content = 'width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no'>
  <meta http-equiv = 'X-UA-Compatible' content = 'IE=edge'>
<meta property = 'og:locale' content = 'en_US' />
<meta property="og:type" content="article">
<meta property = 'og:title' content = 'Flask主线程以及短链接实现' />
<meta name="description" content="问题是这样的：开始有一个功能因为处理流程比较复杂，中间需要对接其他平台，自然在整个访问的稳定性以及访问延迟难以保障，甚至还会出现http连接错误，已经导致服务变慢，其他服务也受到影响不能正常稳定提供服务。此外在高并发下，这个功能业务是需要保证在同一事务下执行。">
<meta property = 'og:description' content = '问题是这样的：开始有一个功能因为处理流程比较复杂，中间需要对接其他平台，自然在整个访问的稳定性以及访问延迟难以保障，甚至还会出现http连接错误，已经导致服务变慢，其他服务也受到影响不能正常稳定提供服务。此外在高并发下，这个功能业务是需要保证在同一事务下执行。'>
<meta property = 'og:url' content = 'https://jxiaof.com/post/flask01/' />
<meta property = 'og:image' content = 'https://jxiaof.com/images/fast-lane.jpg'/>
<meta name = 'twitter:card' content = 'summary_large_image' />
<meta name = 'twitter:creator' content = ''>
<meta name = 'twitter:title' content = 'Flask主线程以及短链接实现' />
<meta property = 'twitter:description'  content = '问题是这样的：开始有一个功能因为处理流程比较复杂，中间需要对接其他平台，自然在整个访问的稳定性以及访问延迟难以保障，甚至还会出现http连接错误，已经导致服务变慢，其他服务也受到影响不能正常稳定提供服务。此外在高并发下，这个功能业务是需要保证在同一事务下执行。'/>
<meta name = 'twitter:image' content = 'https://jxiaof.com/images/fast-lane.jpg' />
<link rel='apple-touch-icon' sizes='180x180' href='https://jxiaof.com/images/icons/apple-touch-icon.png'>
<link rel='icon' type='image/png' sizes='32x32' href='https://jxiaof.com/images/icons/favicon-32x32.png'>
<link rel='icon' type='image/png' sizes='16x16' href='https://jxiaof.com/images/icons/favicon-16x16.png'>
<link rel='manifest' href='https://jxiaof.com/images/icons/site.webmanifest'>

  <link rel='canonical' href='https://jxiaof.com/post/flask01/'>
  <link rel = 'stylesheet' href = 'https://jxiaof.com/css/styles.b934c9412cc837b60543673c5f879b59569b1c7bea59b56896858a3b0a15220c8dbd91e80317b591a1857e31aa26b3bf8fbaca17c5bab12ff1231a272cd6bd6d.css' integrity = 'sha512-uTTJQSzIN7YFQ2c8X4ebWVabHHvqWbVoloWKOwoVIgyNvZHoAxe1kaGFfjGqJrO/j7rKF8W6sS/xIxonLNa9bQ=='>
</head>

  <body><div class = 'nav-drop'>
  <div class = 'nav-body'>
      <a href = 'https://jxiaof.com/about/' class = 'nav_item'>About</a>
    <div class = 'nav-close'></div>
  </div>
</div><header class = 'nav' >
  <nav class = 'nav-menu'>
    <a href='https://jxiaof.com/' class = 'nav-brand nav_item'>江小凡的博客</a>
    <div class = 'nav_bar-wrap'>
      <div class = 'nav_bar'></div>
    </div>
  </nav>
</header>


    <main>
<section class = 'post_header' style = 'background-image:url(https://jxiaof.com/images/fast-lane.jpg);'>
  <h1 class='post_title'>Flask主线程以及短链接实现</h1>
</section>
<div class = 'post'>
  <article class='post_content'><p>问题是这样的：开始有一个功能因为处理流程比较复杂，中间需要对接其他平台，自然在整个访问的稳定性以及访问延迟难以保障，甚至还会出现http连接错误，已经导致服务变慢，其他服务也受到影响不能正常稳定提供服务。此外在高并发下，这个功能业务是需要保证在同一事务下执行。</p>
<p>我认为这个服务可以解耦合，情况比较紧急，所以我起草了一个方案提交通过之后就开始动手了。首先我把这个功能的承载业务的入口和返回结果的出口函数给分离开，使用kafka作为队列保存业务信息，用户提交信息既可。kafka队列实时监听，获取到消息之后就开始处理业务函数保证一个事务一个锁执行。后台处理也可以允许更长时间的轮询。但是这里我把消费者与生产者放到一个服务中，原本不会有问题，在测试环境一切正常。为保证稳定性，我自己还做了一下压力测试，显示正常稳定。</p>
<p>于是，上线开始服务，问题出现了。消费者开始工作了一会儿，没有消费了，kafka待消费数量开始上涨，这个情况开始出现一刹那，就感觉不对。于是，查看代码，添加检索日志，更新在测试环境，测试环境依旧没有问题。上传代码，上线，最后加一句关键的sleep(0.03)（kafka python 启动蜜汁操作）。查看后台kafka数据，果然开始又开始消费了，比之前好一些，从之前断开的偏移量开始，数据没有丢失，也没有重复消费。但是，一个时间点消费到零之后，消费者不消费了，处于一个假死状态。</p>
<p>我怀疑是监听线程被杀死，导致这个监听topic的功能没能进行下去，业务代码比较简单。看了很多次没发现异常，使用线程保护设置守护线程依旧不行，网上看了一下相关问题，没有找到什么答案。继续解耦合，另起一个服务，专门处理kafka消费，彼此独立。编写jobs，docker file ，提交代码。后台实际还是以flask作为web服务，但不需要在启动app实例。上线测试，还是有问题，不消费。说明根源就是线程被kill，设置主线程在main下执行。编写docker file ，更改为python启动， 设置gunicorn worker数量和kafka一致。打包上线，消费者正常了！</p>
<p>之后整理了一下思路，出现的问题原因如下：</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-python" data-lang="python"><span style="color:#66d9ef">if</span> __name__ <span style="color:#f92672">==</span> <span style="color:#e6db74">&#39;__main__&#39;</span>:
    t <span style="color:#f92672">=</span> Thread(target<span style="color:#f92672">=</span>run_schedule)
    t<span style="color:#f92672">.</span>start()
    context <span style="color:#f92672">=</span> (<span style="color:#e6db74">&#39;cert.pem&#39;</span>, <span style="color:#e6db74">&#39;key.pem&#39;</span>)
    app<span style="color:#f92672">.</span>run(host<span style="color:#f92672">=</span><span style="color:#e6db74">&#39;0.0.0.0&#39;</span>,port<span style="color:#f92672">=</span><span style="color:#ae81ff">8080</span>,debug<span style="color:#f92672">=</span>False,ssl_context<span style="color:#f92672">=</span>context)


<span style="color:#66d9ef">if</span> __name__ <span style="color:#f92672">==</span> <span style="color:#e6db74">&#34;__main__&#34;</span>:
    main_thread_pool <span style="color:#f92672">=</span> ThreadPoolExecutor(<span style="color:#ae81ff">10</span>)
    main_thread_pool<span style="color:#f92672">.</span>submit(worker)
</code></pre></div><p>run_schedule和worker将函数永远循环，偶尔解除阻塞以执行任务。</p>
<p>显然，当我不再调试时，使用类似gunicorn的wsgi服务器，它会通过main()调用模块，因此线程无法启动。将它们放在该块之外是行不通的，因为如果任何其他代码导入模块，线程就会启动！</p>
<hr>
<h1 id="短链接">短链接</h1>
<blockquote>
<p>web短链接服务其实实现起来比较简单，提供了后台访问，自行访问。访问如下：就可以了！（新域名：soovv.com （索夫） 还未备案！）</p>
</blockquote>
<p><img src="https://i.loli.net/2020/09/21/wWAU9V4YsDdnaob.png" alt="img"></p>
<p>这里如果使用我的域名,将为用户生成类似:<a href="http://106.54.227.26:5000/b/"><strong>http://106.54.227.26:5000/b/</strong></a>这样的URL.但现在没完成备案,就只能IP访问,看起来怪怪的,这个URL一点也不短.</p>
<p>ps:  <a href="http://182.254.159.57">http://182.254.159.57</a>是一台我练习的开发机器(开放了80端口,同样可以访问首页,其他被我隐藏了).</p>
<p><strong>106.54.227.26 这台主机已被释放,新主机:  <a href="http://182.254.159.57">http://182.254.159.57</a></strong></p>
    <div class = 'post_extra'><div class = 'copy' data-share = 'Share Story' data-copied = 'Link Copied'>
  <svg>
    <use xlink:href="#copy"></use>
  </svg>  
</div>

    </div>

  </article>
  <div class="post-comment">
       
       <h3>Comments welcome !</h3>



<div id="vcomments"></div>
<script src="//cdn1.lncld.net/static/js/3.0.4/av-min.js"></script>
<script src='//unpkg.com/valine/dist/Valine.min.js'></script>
 <script type="text/javascript">
   new Valine({
       el: '#vcomments' ,
       appId: 'xb5Xl8O60fXEK4d13KsazkkL-gzGzoHsz',
       appKey: 'qyztiJ2tgf6AhvsrH088N29o',
       notify: 'false', 
       verify: 'false', 
       avatar:'mm', 
       placeholder: '说说你的看法吧...',
       visitor: 'true'
   });
 </script>
  </div>
  <aside><h3></h3>
<ul class='posts aside'>
<li class = 'post_item'>
  <a class = 'post_card' href='https://jxiaof.com/post/first/' title = 'ABOUT MYSELF' style = 'background-image: url(https://jxiaof.com/images/speakers.jpg);'>
  </a>
  <div class = 'excerpt'>
    <div class = 'excerpt_meta'><div class = 'copy' data-share = 'Share Story' data-copied = 'Link Copied'>
  <svg>
    <use xlink:href="#copy"></use>
  </svg>  
</div>

    </div>
    <h3 class = 'post_link'>
      <a href='https://jxiaof.com/post/first/'>ABOUT MYSELF</a>
    </h3>
    <p class = 'pale'><p>​        I am a light game enthusiast with poor gastrointestinal function. I go to bed …</p></p>
  </div>
</li>

<li class = 'post_item'>
  <a class = 'post_card' href='https://jxiaof.com/post/leetcode/' title = 'Leetcode' style = 'background-image: url(https://jxiaof.com/images/programmer.png);'>
  </a>
  <div class = 'excerpt'>
    <div class = 'excerpt_meta'><div class = 'copy' data-share = 'Share Story' data-copied = 'Link Copied'>
  <svg>
    <use xlink:href="#copy"></use>
  </svg>  
</div>

    </div>
    <h3 class = 'post_link'>
      <a href='https://jxiaof.com/post/leetcode/'>Leetcode</a>
    </h3>
    <p class = 'pale'>买卖股票简答:
class Solution: def maxProfit(self, prices): &#34;&#34;&#34; :type prices: …</p>
  </div>
</li>

</ul>

  </aside>
</div>
<script src = 'https://jxiaof.com/js/autosize.min.js'></script>
<script src = 'https://jxiaof.com/js/timeago.js'></script>
    </main><svg width="0" height="0" class="hidden">
  <symbol viewBox="0 0 699.428 699.428" xmlns="http://www.w3.org/2000/svg" id="copy">
    <path d="M502.714 0H240.428C194.178 0 153 42.425 153 87.429l-25.267.59c-46.228 0-84.019 41.834-84.019 86.838V612c0 45.004 41.179 87.428 87.429 87.428H459c46.249 0 87.428-42.424 87.428-87.428h21.857c46.25 0 87.429-42.424 87.429-87.428v-349.19zM459 655.715H131.143c-22.95 0-43.714-21.441-43.714-43.715V174.857c0-22.272 18.688-42.993 41.638-42.993l23.933-.721v393.429C153 569.576 194.178 612 240.428 612h262.286c0 22.273-20.765 43.715-43.714 43.715zm153-131.143c0 22.271-20.765 43.713-43.715 43.713H240.428c-22.95 0-43.714-21.441-43.714-43.713V87.429c0-22.272 20.764-43.714 43.714-43.714H459c-.351 50.337 0 87.975 0 87.975 0 45.419 40.872 86.882 87.428 86.882H612zm-65.572-349.715c-23.277 0-43.714-42.293-43.714-64.981V44.348L612 174.857zm-43.714 131.537H306c-12.065 0-21.857 9.77-21.857 21.835s9.792 21.835 21.857 21.835h196.714c12.065 0 21.857-9.771 21.857-21.835 0-12.065-9.792-21.835-21.857-21.835zm0 109.176H306c-12.065 0-21.857 9.77-21.857 21.834 0 12.066 9.792 21.836 21.857 21.836h196.714c12.065 0 21.857-9.77 21.857-21.836 0-12.064-9.792-21.834-21.857-21.834z"
    ></path>
  </symbol>
  <symbol viewBox="0 0 60.015 60.015" xmlns="http://www.w3.org/2000/svg" id="reply">
    <path d="M42.007 0h-24c-9.925 0-18 8.075-18 18v14c0 9.59 7.538 17.452 17 17.973v8.344a1.694 1.694 0 0 0 1.699 1.698c.44 0 .873-.173 1.198-.498l1.876-1.876C26.708 52.713 33.259 50 40.227 50h1.78c9.925 0 18-8.075 18-18V18c0-9.925-8.075-18-18-18zm16 32c0 8.822-7.178 16-16 16h-1.78c-7.502 0-14.556 2.921-19.86 8.226l-1.359 1.359V44a1 1 0 1 0-2 0v3.949c-8.356-.52-15-7.465-15-15.949V18c0-8.822 7.178-16 16-16h24c8.822 0 16 7.178 16 16v14z"></path>
  </symbol>
</svg>
<footer class = 'footer'>
  <div class = 'footer_inner wrap pale'>
    <p>&copy;&nbsp;<span class = 'year'></span>&nbsp;江小凡的博客.
    Designed by  <a href = 'https://www.github.com/jxiaof' title = 'Linkedin Profile'>江小凡</a></p>
  </div>
</footer>
<script src = 'https://jxiaof.com/js/index.min.c3b0a769e56310d4359f30762ab615db8cd6ce33d555ee7f68d25e8a40c63ce18bf6b273d35a9b5c033a7f6e71b1d424e74d9f542d93482215c671b3f31f885f.js'></script>

  </body>
</html>
